Causal ML

• The goal is to estimate the causal parameter using ML
  • and obtain valid confidence intervals,
  • while allowing flexible ML methods to estimate some nuisance functions.
• This is particularly attractive when:
  • we have high-dimensional confounders, and/or
  • we want non-parametric and flexible models.

Debiasing

• The debiased version of the moment is of the form:

\[ \theta = E\bigl[ m(W; \gamma) \;+\; \alpha(X)\,\bigl(Y - \gamma(X)\bigr) \bigr]. \]

• We want to find the correct term of \(\alpha\) that removes the first order plugin bias (Robins et al 1994, Chernozhukov et al 2018)

Riesz representer \(\alpha(X)\)

• The function \(\alpha\) is the Riesz representer of the linear functional \(E[m(W; \gamma)]\).

• We focus on problems where there exists a square-integrable random variable \(\alpha(X)\) such that: \[ E\bigl[m(W; \gamma)\bigr] \;=\; E\bigl[\alpha(X)\,\gamma(X)\bigr], \quad \text{for all } \gamma \text{ with } E\bigl[\gamma(X)^2\bigr] < \infty. \]

• By the Riesz representation theorem, such an \(\gamma(X)\) exists if and only if \(E[m(W; \gamma)]\) is a continuous linear functional of \(\gamma\).

Asymptotic theory

• Let \(\gamma_0\) and \(\alpha_0\) be the true nuisance functions, and define the score \[ \phi(W) \;=\; m\bigl(W;\gamma_0\bigr) \;+\; \alpha_0(X)\,\bigl(Y - \gamma_0(X)\bigr) \;-\; \theta_0. \]
• Under standard regularity conditions:
  • 1. Neyman orthogonality of the score,
  • 2. Sample-splitting for \(\hat{\gamma}\) and \(\hat{\alpha}\),
  • 3. Rate condition We have \[ \sqrt{n}\,\bigl(\hat{\theta} - \theta_0\bigr) \;\xrightarrow{d}\; N\!\left(0,\,\sigma^2\right), \]

Implementation

• Partition the set of data indices \(1,\ldots,n\) into \(L\) disjoint subsets of about equal size \(\{I_\ell\}_{\ell=1}^L\).

• For each data fold \(\ell = 1,\ldots,L\):

  • Estimate \(\hat{\gamma}_\ell \in \mathcal{G}_n\) as a nonparametric regression of \(Y\) on \(X\), using observations not in \(I_\ell\).

  • Estimate the debiasing function \(\hat{\alpha}_\ell\) using observations not in \(I_\ell\) by minimizing a sample version of the objective function \[ \hat{\alpha}_\ell = \arg\min_{\alpha \in \mathcal{A}_n} \Biggl\{ \sum_{i \in I_\ell} \Bigl[ -\,2\,m(W_i;\alpha) + \alpha(X_i)^2 \Bigr] + \Lambda_r(\alpha) \Biggr\}, \]

Implementation

• Estimate the parameter of interest using the cross-fitting and debiasing function in the moment function of equation: \[ \hat{\theta} = \frac{1}{n}\sum_{\ell=1}^L \sum_{i \in I_\ell} \Bigl[ m\bigl(W_i;\hat{\gamma}_\ell\bigr) + \hat{\alpha}_\ell\bigl(X_i\bigr)\bigl\{Y_i - \hat{\gamma}_\ell\bigl(X_i\bigr)\bigr\} \Bigr]. \]

• Estimate the standard error of \(\hat{\theta}\) as \(\sqrt{\hat{V}/n}\), where \[ \hat{V} = \frac{1}{n}\sum_{\ell=1}^L \sum_{i \in I_\ell} \Bigl[ m\bigl(W_i;\hat{\gamma}_\ell\bigr) + \hat{\alpha}_\ell\bigl(X_i\bigr)\bigl\{Y_i - \hat{\gamma}_\ell\bigl(X_i\bigr)\bigr\} - \hat{\theta} \Bigr]^2. \]

Simulation

• We generate 1,000 datasets, each with 2,000 observations.

• Covariates:\(Z_1, Z_2, Z_3 \sim N(0,1).\)

• True propensity:\(\pi(Z) = \operatorname{expit}\!\left( (2Z_1 - Z_2 + 0.5 Z_3)\right).\)

• Treatment variable:\(A \sim \mathrm{Bernoulli}\bigl(\pi (Z))\bigr).\)

• Outcome model with known ATE:\(Y = 2 A + Z_1 - Z_2 + 0.5 Z_3 + \varepsilon,\qquad\varepsilon\sim N(0,1).\)

Results

Summary & Future work

• autoDML automatically learns the Riesz representer. This greatly simplifies implementation for complex causal parameters.

• More stable than standard AIPW estimator

• Simple simulations show that learned Riesz weights reduce variance relative to AIPW & TMLE with plug-in models.

• Future work will evaluate autoDML in more realistic simulation studies that are directly motivated by cohort studies.